Storing XML Documents in Databases

INTRODUCTION Ever since the Extensible Markup Language (XML) (W3C, 1998b) began to be used to exchange data between diverse sources, interest has grown in deploying data management technology to store and query XML documents. A number of approaches propose to adapt relational database technology to store The advantage is that the XML repository inherits all the power of mature relational technology like indexes and transaction management. For XML-enabled querying, a declarative query language (Chamberlin et al., 2001) is available. Traditionally, database technology has been offering support for processing large amounts of data. Recent research has provided valuable insights into the nature of semistructured and XML data and has attempted to integrate them into existing paradigms. However, there are still challenges that have to be met to scale XML databases up to production levels as achieved by rela-tional engines and, thus, to gain acceptance among practitioners. Naturally, XML warehouses inherit the power of relational warehouses (Roussopoulos, 1997), but they also face the same challenges; in particular, update and consistency problems of materialized, replicated, and aggregated views over source data need to be solved. This article discusses techniques related to loading XML documents into a document warehouse. All techniques build on well-understood relational database technology and enable efficient management of large XML repositories. To get the most of relational database systems, we propose to do away with the pointer-chasing tree traversing operations, which many applications generate in the form of edit scripts and replace them with set-oriented operations. Edit scripts (Chawathe et al., 1996; Chawathe & Garcia-Molina, 1997) have been long known in text databases and are similar in behavior to Document Object Model (DOM) (W3C, 1998a) travers-als, which are standard in the XML world; they tend to put relational technology at a disadvantage due to their excessive use of pointer-chasing algorithms. We investigate the use of these scripts and propose alternative strategies for cases when they perform poorly. A more detailed description of our experiments is found in Schmidt and Kersten (2002). As we benchmarked the system's performance, it turns out that the use of edit-scripts is only sensible if they only update a rather small fraction of the database; once a certain threshold is exceeded, the replacement of a complete database segment is preferable. We discuss this threshold and try to quantify the trade-off for our example document database. The application scenario which motivates our research consists of a set …